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ABSTRACT 
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adult and community learning 
quality support programme 



A 3-year programme to support ACL providers 
to meet quality requirements of inspection and 
funding agencies and improve their provision. It is 
run by the Learning and Skills Development Agency 
(LSDA) in partnership with the National Institute 
of Adult and Continuing Education (NIACE) and is 
funded by the Department for Education and Skills 
(DfES). The programme includes an advice and 
information service, a website, quality improvement 
networks, staff development workshops and consul- 
tancy, development projects and case studies. 
Details of the programme, extra copies of this guide 
and back-up materials are available on the website 
www.qualityACL.org.uk 

Further guides and workshops are planned on : 

■ equal opportunities 

■ involving part-time staff in the quality agenda 

■ fit-for-purpose systems for small providers 

■ measuring achievement in non-accredited learning. 
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Ask adult and community learning (ACL) practitioners why they should 
listen to learners, and they might say : 

■ the quality of the learning programme is enhanced when learners 
are involved 

b funders and inspectorates demand it 
bo it helps develop new courses 
b it heads off serious complaints 
b it identifies new markets 
□ it helps widen participation 
b it improves retention 

0 it is integral to the process of teaching and learning. 

Whether practitioners are specialists in basic skills, outreach, or 
working with people with disabilities or other forms of disadvantage, 
listening to learners is fundamental to their work. 

Listening to learners is high on the post-16 raising standards agenda. 
This booklet looks at the role of learner feedback in the quality 
improvement process. It suggests how ACL providers can adapt and 
improve their practice to meet the needs of learners in the changed 
policy context. 

The focus here is on local authority ACL. Therefore, much of the 
material draws on, and is influenced by, other quality assurance 
issues in local government, particularly 'Best Value'. 

The term ’provider' is used throughout this booklet to refer to 
an organisation that makes provision directly and/or through 
contracts with others. 
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Why listen to learners? 
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Policy and practice 

The focus of policy and practice should be learners themselves 
and the quality and range of learning opportunities made available 

to them. Fryer 1997, 29 

Recent government policy has attempted to provide a framework in 
which best practice in adult and community learning (ACL) becomes 
the norm. Publications and discussion papers from government, 
funding agencies and inspectorates have a common theme : 
that the learner is central to the quality improvement process. 

For this government, ACL policy is no different from policy in 
other areas of public service : the Local Government Act, based on 
the Modernising Local Government White Paper (1999), established 
common quality assurance processes within the context of 
'Best Value’. 

'Best Value’ is a system whereby public sector bodies have a 
'duty to deliver services to clear standards (quality and cost) by 
the most effective, efficient and economical means available, taking 
full account of equalities and environmental factors' (Cox 2001 , 6). 
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Best Value Review is a process applying to all local authorities. 

There are four elements : 

s consultation (consulting with the local community, as well as 
with learners) 

o comparison (benchmarking against other ACL providers) 

a competition (checking how responsive the service is to 
the local market) 

0 challenge (considering alternative or innovative ways to 
deliver the service). 

Best Value is the single most important framework by which to 
judge the success of ACL providers within the local authority context. 
It is the only quality model to be named in the Common Inspection 
Framework (CIF), the document that sets out the principles applicable 
to the inspections of post-16 non-higher education under part III of 
the Learning and Skills Act 2000. 



Defining 'consultation' 

For many people, consultation means public meetings, talking 
to interested groups, and perhaps leaflets to all households. 

A broader view is taken here, to include all forms of communication, 
'dialogue' and involvement: giving information, finding out about 
people and what they think, and involving them directly in 
decision making. www.idea.gov.uk/bestvalue/consult 

So consultation in local government is much closer to the concept 
of ’learner involvement' than ’learner feedback'. The concept of 
involving learners (and others) in all aspects of the service that affects 
them is reflected in government publications on quality in ACL. 
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Putting learners at the heart of the system 

The Fryer Report (1997, 25) used the phrase 'putting learners at 
the centre' of the learning process. This was not only about learners 
taking responsibility and having ownership of their own learning 
journeys, but also about widening participation in adult learning. 
Best practice has shown that listening to current learners helps to 
formulate strategies for engaging non-participants (McGivney 2000, 9). 

In terms of the quality improvement agenda, Raising standards 
in post-16 learning refers to 'placing the learner at the heart of 
the system' (DfEE March 2001 , 10) . In practice this means that : 

b ACL providers need to engage with learners as a key element 
of their quality improvement strategy and to show that they have 
done so effectively 

□ local and national learning and skills councils will engage with 
learners as a key element of their quality improvement strategy 
(LSC 2001 , 3) 

o inspectorates will engage with learners as part of the 
inspection process. 

Later in this booklet, we will look at how ACL providers can engage 
with learners effectively (see 'Listening effectively,' page 17). 



Self-assessment 

The process of self-assessment is not new to ACL providers. For 
example, adult education services in local authorities are familiar 
with the practice of ongoing evaluation, monitoring and review at 
a variety of organisational levels - authority-wide, whole service, 
institution, centre, course and individual. However, this process 
needs to be distilled into a new format. This format will influence 
the processes involved. 



Why listen to learners? 5 




-C 

x 



Central to the inspection process and the format of the 
inspection reports are the seven key questions underpinning 
the Common Inspection Framework: 

1 How well do learners achieve? 

2 How effective are teaching, training and learning? 

3 How are achievement and learning affected by resources? 

4 How effective are the assessment and monitoring of 
learners' progress? 

5 How well do the programmes and courses meet the needs 
and interest of learners? 

6 How well are learners guided and supported? 

7 How effective are leadership and management in 
raising achievement and supporting all learners? 

Draft guidance on what these questions mean in the context 
of ACL has been published by the Adult Learning Inspectorate. 

The guidance identifies how listening to learners can assist ACL 
providers in answering these questions. For example, in terms 
of leadership and management (question 7), providers will be 
required to: 

ea ensure all learners receive a statement of their Learning Entitlement 
and understand their responsibilities as learners ... ( arrangements 
for receiving feedback, including complaints, from learners 
and customers will be established for both the LSC [ Learning and 
Skills Council] and providers) 

a have effective systems for gathering feedback, including 

customer surveys, learner satisfaction surveys and leaver surveys 

and act upon the feedback to make improvements 

□ demonstrate financial probity and value for money by ... 

gathering feedback from customers, learners and employers, 

to ascertain that they are providing value for money. 

DfEE 2001, 13-14 
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The following diagram outlines the process of self-assessment. 
It shows where the guidance about involving the learner has an 
impact on the process. 



Listening to learners in 
the self-assessment process 
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Summary 

Listen to learners to improve the service provided to them and 
to others. In doing so, and in approaching the process in a certain 
way, we will better meet the requirements of funding agencies 
and inspectorates. 



Key points 

1 One of the central thrusts of government policy is that people 
have a right to comment on and influence public services. 

2 The learner is central to the quality improvement process 
in adult community learning. 

3 Involve the learner in the planning and development of services 
and not just in giving feedback on provision. 

4 Present evidence of the learner's involvement. 

5 Funders and inspectors also seek to engage directly with learners. 

6 Learner involvement should lead to action. 
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Who is the 'learner'? 



Who should ACL providers involve in their quality improvement 
processes? 

The Best Value process differentiates between 'customers' and 
'beneficiaries'. The table below (adapted from Best Value consultation 
guidance prepared by the Improvement and Development Agency) 
provides a useful framework against which ACL providers can make 
judgements about who should be involved in their self-assessment 
processes. 



Best Value review : who to consult 

Customers 

a Current users of the service 

□ Paid for at point of service delivery 

□ Paid for indirectly 

Consumers of 'collective goods' (eg parks, clean air) 

0 Non-users of the service 

□ Dissatisfied - tried and gone away 

□ Not heard about it 

□ Not yet appropriate (eg 'what would you expect from 
an ACL service if you did use it?') 

□ 'Denied customers ' - those who want to use services 
but are not allowed to or are unable to access them 

0 Internal customers for services provided by one part of 
the council for another 
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Beneficiaries of services 

□ Eg parents or employers as the beneficiaries of the education system 

Other stakeholders 

a Others involved in providing services, eg carers or council officers 
0 Citizens in their role as electors, council tax payers and residents 

Proxy consultees 

0 Those speaking on behalf of others, eg on behalf of very young 
children. This ensures the inclusion of those for whom traditional 
consultation is hard, eg students with learning difficulties. 

www.ideabestvaiue.net 

Your immediate priority may be attaining current learners' involve- 
ment through feedback, but you can also gain a wider perspective 
on the quality of your service and how it may be improved. 



Customers and learners 

The importance of a 'satisfied customer’ is regarded as central 
to success in both the private and public sectors. This is not merely 
so that existing 'customers’ continue to buy the 'product', but so that 
the product itself is improved and more people buy it or it appeals 
to new markets. 

The ’customers' of adult community learning are : 

0 external funders (agencies with whom you have a contract) 
b internal funders (another part of your organisation, which funds you) 

b the general public (who expect a service to be available, 
even if they do not use it) 

0 local communities and employers (and their representatives) 
n your staff (who invest time and energy in your work) 
o priority or target groups of learners (who may or may not attend) 
o your current learners (on learning programmes). 
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Who to involve 



In considering who to involve in the quality improvement process, 
think about who your services affect most directly. 

You may decide that your ACL provision has most impact on your 
current learners and that the majority of your funding is aimed at 
teaching and learning interactions. If so, prioritise the involvement 
of current learners. 

However, focusing solely on current learners to the exclusion of 
other 'customers', 'beneficiaries', 'stakeholders' or 'proxy consultees' 
would not meet the requirements of self-assessment. Some of these 
service users' may not be current learners (eg non-returners or 
non-completers) or they may be 'potential learners' from target groups. 

In each case, seek feedback on specific issues (eg reasons for 
non-completion) and on general issues (eg accessibility or perceived 
barriers to learning). 
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case study Service users who are not current learners 



m Rod rang the service to find out about the nearest Level 3 German 
course to his workplace. He was given impartial information and 
advice that allowed him to choose a suitable programme offered 
by another organisation. 

n Leila is the full-time carer of Hasim, an adult with a learning disability, 
who attends one of the service's self-advocacy courses. 

o Esme runs a small voluntary organisation. She has encouraged 
and supported several members of her team to attend literacy 
and numeracy courses run by the service. 
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Grouping and prioritising 

Before planning the design or implementation of learner feedback 
mechanisms, consider the target audience and the purpose of 
seeking feedback from them. 

You may find it useful to group people from whom you want to obtain 
feedback so that you can target methods accordingly and so that 
you consult a wide cross-section of people. 

It is useful to look at four groups : current learners ; previous learners 
potential learners; and other service users. Within each group, 
sub-groups can be identified to increase the sophistication of 
targeting and general planning. (This could be regarded as evidence 
under the 'leadership and management' strand of the Common 
Inspection Framework, see page 6.) It is possible for an individual 
to be a member of more than one group. 

Current learners : 
a in programme areas 

m who have a disability or a learning difficulty 

ei from ethnic minority groups 

b on different sites 

ei in smaller community venues 

a for whom English is not a first language 

0 who had a pre-course interview 

s who attended an open day. 

Previous learners : 
a in programme areas 

Q who have a disability or a learning difficulty 

□ from ethnic minority groups 
b on different sites 

0 in smaller community venues 
b for whom English is not a first language 

□ who had a pre-course interview 

a who responded to a 'drop-out letter’ from their tutor 

□ who left for known reasons 

□ who left for unknown reasons 

□ who attended one session only. 

Who is the 'learner ? 13 
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Potential learners : 



□ for programme areas 

□ who have a disability or a learning difficulty 
m in six target wards 

m for whom English is not a first language 
b who had a pre-course interview 
a who attended an open day 
a who attended 'taster' sessions 
0 from ethnic minority groups 
m who are older people. 

Other service users : 

ej who are parents or carers of learners in programme areas 
b who are advocates of adults who have learning difficulties 
b who provide complementary services in health or social care 

□ who are employers of current learners 
o who are employers of previous learners 
B who are employers of potential learners 
b who represent target groups of learners. 

Grouping is a cost-effective way of managing the feedback process, 
ensuring that all learners are consulted at some stage. Targeting is 
more inclusive than a random ’scattergun’ approach. 
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Summary 

Be proactive about why you want to involve learners and how. 
Target resources to gather evidence for the process and results 
of listening to learners. 



Key points 

1 Consider what you mean by 'learner' and 'customer' in the context 
of your own service. 

2 Decide who to involve and prioritise groups. 

3 Gather evidence about the process (eg targeting or sampling) 
and how it is applied. 

4 Link target groups to inspection or quality frameworks 
as soon as possible. 
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Listening effectively 
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Why use frameworks? 

Listening to learners is not an end in itself. In terms of both proving 
and improving what ACL services do, it provides essential evidence 
of rigour in the self-assessment process and in the provision itself. 

It also enhances the scope and quality of the service. Many providers 
adopt external quality frameworks that meet the needs of the whole 
organisation, even though only a part of their work may be in ACL. 

Providers have chosen frameworks because they provide quality 
criteria against which the whole organisation can judge or measure 
itself. Using these quality criteria, actual performance information 
can be gathered and then acted upon for quality improvement. 

Once certain criteria are established, it is easier to define levels 
of performance - norms, strengths and weaknesses - and set 
targets for improvement (see Dixon and Moorse 1998, 3). 

Off-the-shelf quality frameworks are useful as they provide 
consistent standards against which to judge performance. 

In addition, good practice can be shared, both in terms of 
performance and the way that it is measured. 



What frameworks are there? 

Appendix 1 summarises what is available. 
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Being specific 

Frameworks are generally felt to be useful because they help translate 
a quality policy into quality improvement practice. But to be effective 
any framework, system, manual or handbook has to enable you 
to focus specifically on what you need to know in order to improve. 

Use general quality criteria to define what good practice means 
to you and develop your own set of quality standards against which 
to assess the quality of your service. 

Providers using existing quality frameworks will be adapting 
their practice in the light of the Common Inspection Framework 
and subsequent guidance. 

For listening to learners to have a purpose and result in action, 
the following five-step sequence is important : 
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Feedback 



The most effective frameworks 'complete the loop' of the listening 
process by reporting back to those who took part in the consultation. 
Feedback is important, even if you have not taken action as a result 
of the listening consultation ; the essential thing is to have listened. 
There may be good reasons why you have not taken action and 
you should explain these. Besides, people are more willing to give 
feedback a second time if they know it was effective the first time. 



Setting standards 

Some providers use a student charter or statement of learner entitle- 
ment against which to measure themselves. These are likely to follow 
the Common Inspection Framework or other quality frameworks that 
the provider is using, but sometimes their content or presentation 
is adapted to suit the target audience of learners. 

An example of good practice is the Our right to learn charter 
for adults with learning difficulties (Jacobson 2000). 

See also the Adult Learning Inspectorate's adaptation of question 5 
of the Common Inspection Framework, below : How well do the 
programmes and courses meet the needs and interest of learners ? 



Common Inspection 
Framework 



Learners have the opportunity 
to study an appropriate range 
of courses or programmes, 
and, where appropriate, 
achieve suitable qualifications 



In adult and 
community learning 
this means : 

also, the timing of courses 
is designed to fit with adults ' 
other commitments eg shift 
working, carer responsibilities 



Adult Learning Inspectorate February 2001 , 68 



Now plan to listen to learners to find out whether your service 
is meeting this standard. 
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Planning to listen 

Ask yourselves three questions : 
d What do we need to know? 

E3 Whom do we need to involve ? 

0 How do we go about it? 

What do we need to know? 

□ Do existing learners think courses fit in with other commitments? 

□ Were people who enquired but did not join put off by the timing 
of courses? 

□ Do potential learners think timing is an issue? 

□ Did previous learners drop out because of the timing of courses? 

□ Would 'basic skills in the workplace' learners have attended 
a course outside the workplace? 

□ Did carers of learners find the timing of courses suitable ? 

Whom do we need to involve? 

□ A sample of, or all, current learners? 

□ A sample of, or all, enquirers? 

□ A sample of potential learners? 

□ All previous learners who have dropped out? 

□ A sample of, or all, basic skills in the workplace learners? 

□ A sample of, or all, carers of learners? 

How do we go about it? 

□ Clarify what is meant by adherence to the standard. 

□ Plan how to measure responses. 

□ Decide how to sample. 

□ Combine questions across a number of possible tools. 

□ Select the best tools. 

□ Adapt tools for specific target groups. 

□ Pilot new tools and amend them if necessary. 
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Good planning is about making the best use of resources : 
financial resources, the provider's time and the learner's time. 

Any evidence-gathering should also be ’fit for the purpose' of 
improving the quality of the service. Benefits must outweigh costs. 
Before implementing a plan, be clear about what you want from the 
activity, which target group it is aimed at and the methodologies 
or approaches to be used. 



Implementing 

Taking the single issue of course timing as an example, the 
table below describes possible approaches for obtaining learners' 
(and others') views. In reality, this issue would be assessed at the 
same time as associated quality criteria, such as : 

b accommodation provides a suitable setting for good teaching, 
training & learning and support for learners (CIF 3) 

b programmes of work take account of community & employer needs 
(CIF 5) 

m impartial guidance enables learners to choose the course 
or programme which is right for them (CIF 6) 

m individual learning needs are accurately diagnosed & learners 
receive effective additional support throughout their studies 
or training (CIF 6) 

b learners have effective personal support to help them to 
complete their course or programme, including access to 
specialist support services (CIF 6). 

Combining these questions makes more efficient use of resources 
and recognises the complexity of learner motivation by allowing 
learners (and others) to respond on a number of interlinked issues. 
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case study Assessing course timing 



Action taken 



Resources used 



Question added to the whole-service 
'welcome survey' of learners, 
given to them at their first course 



Question added to information 
and advice survey for a sample 
of all enquirers. Respondents 
asked if they would be willing to be 
interviewed further by telephone 



Short, semi-structured telephone 
interview with small sample of 
respondents above on a number 
of issues, including course timing 



Focus group on course timing 
established at main centre 



Few additional resources - 
the additional question needs to 
be filled out by learners and then 
analysed by a member of staff 



Few additional resources - 
the additional question needs to 
be filled out by learners and then 
analysed by a member of staff 



n Development time 
s Staff training on interview techniques 
0 Piloting time 

□ Staff time (alongside existing duties) 
0 Analysis and reporting time 



b Development time 
h Staff development on 
running focus groups 
m Training for learners on 

running focus groups (alternative) 
b Targeting, recruitment and 

selection time and advertising cost 
b Staff time 

b Refreshments and expenses 
b Analysis and reporting time 
h Room hire 
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Action taken 


Resources used 


Question raised at community 
consultation meetings or any 
external partnership meetings 
that members of the service attend 


0 Development time for 

a common format for question 
□ Briefing for all who are 
going to ask the question 
o Time to analyse responses 
and report back to the group 


Question added to 'drop out’ 
questionnaire, letter and follow-up 
telephone call 


The current system of following up 
non-attendance may have needed 
revising anyway 


Session during basic skills in the 
workplace courses on 'motivation 
for doing this course’ 


0 Curriculum leadership time 
■ Teaching and learning time 
o Reporting back time 


Advertisement or editorial placed 
in a newsletter for carers, advising 
service's free-call internet-based 
feedback site 


Assumes feedback site already 
established. If not, development 
costs £3000 and ongoing costs 
£800 per year. Staff or volunteer 
time to monitor and report. 

Some council sites will already 
have this facility and staffing. 
Cost: liaison time 




Listening effectively 23 




28 



Analysing, evaluating and taking action 

The more complex the means of gathering feedback, the harder 
the task of analysis. Working to national and external quality 
frameworks increases the number of factors to be considered. 

The data must be evaluated : what does it say, and how? 

The way the data presents will allow you to learn how effectively 
you are consulting with learners. This in turn meets the requirements 
of the Common Inspection Framework : 

m The 'best value' principles of comparison, challenge, consultation 
& competition are applied in securing resources & services (CIF 7). 

m The quality assurance arrangements are systematic & informed 
by the views of all interested parties (CIF 7). 

Once all the data is collected, it can lead to action on the provision 
of services to learners and potential learners. It can also influence 
how the quality assurance process is managed and how learners 
are involved. 
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case study Two of the lessons from consultation exercise 
are that: 



□ learners, non-learners, community groups and carers think there 
is not enough one-off, one-day weekend provision of basic skills 
workshops and 'Towards Independence' sessions for adults 
with learning difficulties 

b communication with, and in particular getting the views of, 
adults with learning difficulties is problematical ; it is unclear 
whether the service is getting the adults' own views or those 
of interested parties. 

A development plan is drawn up : 



Action 


Outcome/target 


Further Saturday school provision 
will be developed 


□ 10 Saturday courses developed 
in year 1 

□ 60 students attend these, 

50% not being current learners 


A charter for working with adults with 
learning difficulties will be developed 
to include 'the learner's voice' 


E] Existing good practice is 
researched and summarised 
□ Learners are consulted on 
a draft pack and processes 
b Training is provided 
h Standards are developed 
and published 


Try writing success criteria related to these actions in terms of 
their impact on learners (see Kenway and Reisenberger 2001 , 34) 
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Reporting back 

The success of the next cycle of listening to learners will depend 
on the quality of reporting back this time. Learners are more likely 
to respond next time if they have received feedback on how their 
views have been used. It is important to provide feedback on : 

b what learners' views were and how many shared those views 
0 what you are going to do about it and when 
0 what you are not able to do and why. 

You can use many ways of feeding back to learners and 
the wider community : 

0 student handbooks or diaries 
m brochures or prospectuses 
0 open days or enrolment sessions 
m circulars or newsletters 
b briefings or minutes to meetings 
b reports to official committees and partners 
0 open forums and websites 
□ advertisements and editorials in the press 
0 informal class or breaktime visits 
b roadshows 

b tutor-led communication. 



Investing in the future 

Learner feedback is not just about proving and improving the quality 
of existing provision. It is a method of extending the scope of ACL. 

If you are to engage with potential learners, particularly those who 
are most difficult to reach, you need to engage with them on their 
own terms. 

'We thought it would be like school, we didn't think we'd be treated 
as adults or that people would listen to us,' is a familiar refrain. 

Learner involvement is an essential component of outreach provision 
that starts with listening to the group's needs and leads to the design 
of the learning programme. 
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Summary 

To listen to learners effectively, you must be specific about what 
you want to know. Apply this increased focus across all stages 
of the process. Listening to learners is about increasing both 
the quality and the scope of provision. 



Key points 

1 The process of thinking about listening to learners allows you 
to focus on what you want from it. 

2 Bear in mind the end of the process : reporting back to learners. 

3 Set standards and plan carefully how to measure them. 

4 Use a range of tools. 

5 Use the analysis as a basis for reflection and development planning. 

6 Learners' views are integral to the curriculum development process. 

7 Listening helps develop curricula that are appropriate to the needs 
of current non-participants. 
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Deciding which approaches, or tools, to use will depend on what 
you want to find out. A tool that is good for one job may be useless 
for another, perhaps excellent in terms of potential results but 
too cumbersome and too expensive, ie not 'fit for purpose'. 

The Improvement and Development Agency (IdeA) Best Value website 
(www.ideabestvalue.net) lists over 100 methods of consultation. 
Some of the methods most relevant to ACL are listed below, with 
an analysis of their advantages and disadvantages. 



A range of tools 

Questionnaires and surveys 

Questionnaires and surveys can be conducted by post, by telephone 
or face-to-face. Their scope and impact depend on many factors : 
numbers reached, the nature of the questions, whether they are 
conducted by interviewers or by self-completion. 

The most common approach for ACL providers is to survey 
’student satisfaction' using the Audit Commission performance 
indicator K1 6, although this is not a compulsory Best Value indicator. 
This is unlikely to be sufficient for the current inspection and 
self-assessment processes. 

Surveys tend to be handed out by tutors and returned either via 
the tutor or using a pre-paid envelope or postcard. The returns are 
then collated and analysed centrally, as opposed to being dealt 
with at each centre (involving some coding of response sheets 
if the questionnaire is anonymous). 
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This allows comparisons to be made by centre or by curriculum area. 
One authority has tried giving two questionnaires to each student, 
one being aimed at a friend who is not a current learner. This personal 
approach elicited a higher response rate than previous surveys. 

The challenge lies in making the document as focused, easy to use 
and clear as possible. If there is any ambiguity in the questions, 
then results are skewed. This model also depends on the skills 
and engagement of the front-line staff encouraging the participants 
to fill in the form. Again, results may be skewed or response rates 
affected by how the questionnaire is presented to learners. 

For some groups of learners, such as those with learning difficulties, 
use of a standard form may need to be facilitated by a member 
of staff or an advocate. The advocacy role is considered to be the 
more robust approach, closer to the learner's own views. Similarly, 
beneficiaries other than the learner, such as a parent or day care 
provider, should be surveyed separately. Do not assume that 
beneficiaries' views necessarily reflect those of the learners. 

A questionnaire is often used at the beginning (in a 'Welcome 
survey') or end of the programme of study. One of the criticisms 
learners have of this system is that their ideas tend to 'go into 
a great black hole’ ; they do not know how their feedback was used. 

It is useful therefore to state on the form what the impact of the 
questionnaire will be and when you will report back on results. 

Non-standardised approaches aimed at specific target groups 
may be used alongside the survey, but it is also useful to present 
all learners with the same questions. In such cases you may need 
to amend forms, using language and cultural styles accessible 
to the learner. 
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Advantages Disadvantages 

Questionnaires and surveys Questionnaires and surveys 

b reach large numbers □ 

comparatively inexpensively 0 

□ involve all learners who 0 

use the service 
m provide useful comparative 
information by centre 
or curriculum area 0 

m are specific and quantifiable 
m can be used to benchmark over 
time and with results elsewhere 
m are regarded as inclusive if 
on a large scale. 

Comments, suggestions and complaints schemes 

These include comments postcards or suggestion boxes located in the 
reception area or in classrooms. You need systems for ensuring that 
comments or complaints are dealt with systematically and recorded, 
so lessons can be learnt. See Consultancy for Free (Owen 2001 ) 
for detailed guidance on this aspect of listening to learners. 

Feed back to learners any actions taken as a result of their 
suggestions (unless they are anonymous). A good, well-publicised 
complaints scheme should encourage complaints ; the number of 
complaints received should not be used as an indicator of success. 



can contain unreliable questions 
do not allow for two-way dialogue 
do not allow for discussion or 
deliberation, so are not useful 
in areas where respondents are 
are unlikely to be well informed 
can be regarded by learners 
as a one-way process. 



Advantages 

Comments, suggestions 
and complaints schemes 

0 can result in greater satisfaction 
for compainants if they are 
dealt with properly 
0 can be a source of new ideas 
and creative solutions. 



Disadvantages 

Comments, suggestions 
and complaints schemes 

0 may be unrepresentative 
b disadvantage learners who 
have difficulty with reading 
or with the language used 
0 only work within a culture 
that encourages feedback. 
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Invitations for written inputs 

General requests for comments from the public or from learners 
can be made through local authority newspapers or leaflets, either 
of limited coverage or sent to all households. Requests may simply 
consist of a couple of open or closed questions, linked to a 
competition for a free course. 

This approach gives everyone the chance to have their say. 
Responses take into account information given in the leaflet, 
so participants can be more informed and are able to comment 
specifically and in more detail. Some providers include a 'have 
your say' section in their course brochures. 



Advantages 

Invitations for written inputs 
m are inclusive 

0 show people that you are listening 
m positively affect the 

perception of your institution 
and its reputation 
b are relatively cheap if the 

local authority newspaper is used. 



Disadvantages 

Invitations for written inputs 

b are not totally objective 
□ may be difficult to analyse where 
open-ended questions are used 
(rather than 'tick boxes’) 
b may result in a large number 
of responses. 
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Course-related tools 



Various standard quality assessment approaches are applied 
systematically : learners are asked to keep diaries of their learning 
experiences; tutors' lesson plans may include a learners' views prompt 
in their evaluation section ; non-attendance may trigger a letter to 
the learner; leavers may be invited to attend an exit interview. 



Advantages 

Course-related tools 

® are linked to the learning process, 
as opposed to external to it 
■ provide precise and 
detailed information 
a measure changes in perception 
over time 

a are useful for checking specific 
questions, eg ease of finding 
facilities, treatment by staff, 
quality of learning materials 
and communication 
b in addition, learner diaries 
illustrate real experiences 
at point of use. 



Disadvantages 

Course-related tools 

si are perceived as taking up 
'valuable class time' unless 
directly linked to the 
learning process 
q require staff development on 
how to 'capture the feedback' 
from a range of sources 
a are perceived to impose on 
tutor time 

a in addition, learner diaries may 
not be representative, particularly 
if only used in specific classes 
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Informal contacts 

Informal conversations with learners and others during classrooms 
visits, break-times or open days can be recorded as part of normal 
activity. Formal sampling may then take place, for example looking 
at how learners perceive health and safety issues. This can be 
recorded in the same way as an audit of telephone enquiries, 
with a standard form for each participating staff member. 



Advantages 

Informed contacts 

□ demonstrate genuine listening 
m are a good way of introducing 
and exploring ideas which 
can then be followed up and 
tested more systematically 
0 can be integrated into 
existing practices. 



Disadvantages 

Informed contacts 

□ require effort to record results 
systematically 

0 risk bias as to who is listened to 
and which comments are accepted, 
remembered and recorded 

0 require additional time for 
setting up systems, recording 
and analysing. 



Observation 

This involves observation of the whole service, as distinct from 
the observation of teaching and learning. For example, observing 
how people use a reception desk or counting numbers of potential 
learners taking course information sheets. Direct observation, 
shadowing, cameras or closed circuit television can be used. 

Think through and establish openly the principles of observation, 
particularly when observing people's work behaviour. Staff should 
be consulted before the programme is introduced. 



Advantages 

Observation 

b provides an accurate record of 
people's behaviour, as opposed 
to what they say they will do 
is can be integrated into 
existing practices. 



Disadvantages 

Observation 

Q does not record how people 
feel and think about the service 
b may involve some (probably 
unconscious) observer bias 
m people may not act naturally if 
they know they are being observed. 
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Telephone polling 

Telephone polling can be carried out very quickly, using a short 
form that can be used for follow-up surveys. For example, a brief 



call could be made to those 



Advantages 

Telephone polling 

a can be cheaper than surveys, 
due to lower refusal rate 

0 has the benefit of personal contact 
without the intimacy of being 
face-to-face. 



who enquired but did not enrol. 



Disadvantages 

Telephone polling 

0 excludes those without a telephone, 
who may already be among 
the more disadvantaged 
q can be expensive in staff training 
as this is a skilled activity. 



Citizens' Panels 

Many local authorities now use Citizens' Panels. These are 
authority-wide panels of 1000-2000 community representatives, 
surveyed several times a year, usually by post or telephone. 

You could approach your local authority about adding questions 
to their survey. Whole panels or sub-groups can also be used for 
deliberative workshops or as focus groups. Using the same Panel 
a number of times reduces recruitment costs. 



Advantages 

Citizens' Panels 

m reduce recruitment costs if the 
same panel is used a number 
of times (since people agree 
to be on the panel) and should 
increase response rates, as 
the panel is a captive audience 
0 may allow for some measurement 
of changes over time as the 
same group is being used 
e should be cheaper than 
an equivalent one-off survey. 



Disadvantages 

Citizens' Panels 

m may be unrepresentative as 
people on the panel become 
more knowledgeable 

a may reduce validity of tracking 
over time if there is a high turnover 
of panel members 

b involve varying costs ; they depend 
on size of panel, means of recruiting, 
method of polling, whether costs 
are shared with other agencies. 
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Public meetings 

These one-off forums allow small-group discussions and feedback 
through oral reports and anonymous, informal comments on 
post-it notes and flipcharts. Consider carefully how to promote 
the event. 



Advantages 

Public meetings 

m allow those present to set 
or influence the agenda 
□ give a large number of people 
the chance to have their say, to 
explain and to supply information 
0 have the potential for informed 
discussion and hearing the views 
of a large number of people 
0 show that the provider is listening 
0 may be reported widely in 
local media, reaching those 
who do not attend. 



Disadvantages 

Public meetings 

a can involve problems of 
unpredictable take-up and 
cost-effectiveness 
b need wide publicity to make them 
worthwhile and careful timetabling 
to avoid competition with other 
leisure activities. 
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Forums and interest groups 

Area and neighbourhood forums involve learners (and others) in a 
particular location. They may focus on issues wider than the learning 
community. Membership may be restricted or open to the public. 

A budget may be devolved to the forum. The agenda may be set in 
advance or formulated at the time, according to participant concerns. 

Interest groups or specialist groups are useful for detailed and 
specific feedback relating to particular aspects of ACL provision. 
This includes regular consultation with existing bodies, 
including parish councils, professional bodies, advocacy groups, 
ethnic minority groups, voluntary and advice-giving bodies, 
civic societies, sports and leisure societies and other stakeholders. 
Groups raise issues for providers to address, rather than suggesting 
immediate solutions themselves. 



Advantages 

Forums and interest groups 

b reach people in their own areas 
and address specific 
local concerns 

0 relate to natural communities 

□ are relatively cheap because 
groups already exist and 
have an in-depth knowledge 
of their own community, 
including hard-to-reach groups. 



Disadvantages 

Forums and interest groups 

h may involve a significant budget 
for a small area ; it needs careful 
cost-benefit analysis and 
commitment to keep the 
forum going 

0 may not be representative of 
the community and are unlikely 
to represent non-users 

a involve high costs if the group 
is specially established by 
the ACL service. 
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Focus groups 

Focus groups explore issues through structured but open-ended 
discussion by a group of around 10 people, representative of a 
particular sector, led by a trained facilitator. Facilitation is skilled 
work, requiring either staff training or induction, or the use of 
external consultants. Some providers train groups of learners 
(eg older people) to carry out this work. 

Focus groups can be deliberative : they may learn about, advise 
and even vote on a particular issue. Some providers have consulted 
focus groups about the layout of their brochure, student handbook or 
prospectus; others to inform the development of a curriculum area 
such as basic skills provision. 



Advantages 

Focus groups 

m are good for issues that need 
an in-depth, qualitative review 
b are useful for generating questions 
for quantitative analysis, or in order 
to analyse and explain issues after 
quantitative survey 
□ are effective in assessing 
reaction to proposed changes 
b avoid just hearing 
the 'loudest voices' 
m assist targeting, as they can focus 
on sections of the community 
previously excluded 
b allow ideas through group 
discussion to be built on and 
new directions taken, rather 
than following a single individual's 
view or preset questions. 



Disadvantages 

Focus groups 

0 cannot be guaranteed to be 
statistically representative 
of the whole community 
□ may need to be repeated to cover 
all target groups or a range of issues 
0 can be expensive in terms 
of training or the use of 
external consultants 
b require skilled facilitation. 
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Exhibitions, roadshows and open days 

Take the opportunity to listen, as well as to give out information. 
Collect information through self-completion questionnaires, 
comments cards, questions asked by interviewers or members 
or staff, informal discussions (with main points noted), or through 
comments on 'post-it' notes (which can then be grouped by theme). 
Use display boards, models, written material and video to present 
your information effectively. 

Consider attending other agencies' open days and events, such as 
an Early Years Development and Childcare Partnership information 
session on childcare and playwork training opportunities. Collecting 
demographic information will help determine how representative 
the views of participants are. 
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Electronic communications 

The internet and cable television allow interaction with the public. 
A number of learner forums have developed online sites run by 
and for learners, but accessible by anybody who has access to 
the internet. Check sites to ensure that they are accessible for 
people who have communication difficulties or other disabilities. 

Video box enables the public to leave video messages; 
video conferencing links facilitate direct contact with staff. 
Opinions can also be recorded using conventional audio or 
video recorders. 



Advantages 

Electronic communications 

m are relatively cheap 
o involve learners in setting up 
and running the service 
s are convenient, as access can 
be from own home or library 
o allow provision of information, 
discussion and collection of views 
0 allow extensive documentation 
to be made available on the web 
without large printing costs, and 
allowing selective access 
a increase accessibility of people 
who have limited mobility 
or communication difficulties 
or disabilities. 



Disadvantages 

Electronic communications 

a can be regarded as 'impersonal' 
e can reach only a limited number of 
people have access to the internet 
0 can lead to inaccurate results : 
it may be hard to verify the 
accuracy of respondent details 
(so you may get some multiple replies, 
or replies from outside the area) 
ei often involve high initial 

infrastructure costs, depending 
on baseline capability and access 
to whole authority systems 
0 usually require specialist help 
in designing web pages, discussion 
forums and analysing responses. 
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'Open space' 

In this very flexible approach, those attending an 'open space’ 
workshop determine issues for discussion. Individuals interested 
in particular issues take responsibility for leading a workshop. 
Participants sign up for the workshops, which are recorded 
for feedback at the close of the event or shortly after. 

'Open space’ workshops are good for addressing difficult issues 
involving a large number of people, particularly where there are 
conflicting views. This approach is characterised by flexibility, 
informality and individual willingness to participate. It is a good 
idea to get some experience of the approach before trying it out, 
by observing a workshop elsewhere, for example. You may 
find it stimulating and different. Those who would normally take 
responsibility for running such an event should instead be treated 
as 'just another participant’. 



Advantages 

Open space 

m is flexible, informal and 
empowering 

□ is good for addressing difficult 
issues, involving a large number 
of people, where there are 
conflicting views 

m often results in positive reactions. 



Disadvantages 

Open space 

e requires experience and 
skilled organisation 

0 involves cost of an experienced 
organiser for the day and 
a large venue. 
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Learner nominees 



Many of the methods described above may lead to the identification 
of learners who may be willing to work alongside staff as 'learner 
nominees' to assist specifically in the quality improvement process. 

This approach is currently being piloted by the Adult Learning 
Inspectorate. Early indications are that it presents challenges to 
provider, inspectorate and nominee alike. But the challenge is worth 
undertaking because it shows a willingness to involve learners in 
the quality improvement process and listen to feedback from them. 

Questions yet to be answered are whether the individuals chosen 
are representative of the diverse ’student body' in ACL, and how 
far they can be fully involved in, and knowledgeable of, the issues 
affecting the service. 



Monitoring and evaluation 

Consider how you will evaluate whether you have been effective in : 
m choosing the right approach or tool 
0 applying it (within budget) 

0 providing evidence 
0 linking outcomes with planning 
0 feeding back outcomes to learners. 

There are a number of ways of monitoring the efficiency 
and effectiveness of listening to learners : 

0 always pilot surveys and questionnaires 

b establish a unit cost for the tool used, so it is comparable 
with other methods 

0 define the answers you want from the tool and measure 
how many you have got 

o apply standards for the robustness of evidence 
(in terms of reliability, validity and relevance) 

0 get the views of learners on the listening process ! 
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Summary 

Develop a range of approaches to listen to and involve learners 
as part of the quality improvement process. Use a range of tools 
that are fit for the purpose of obtaining the information required. 
Monitor and evaluate this process in a way that involves learners 



Key points 

1 A learner feedback toolkit is a useful part of a quality system, 
demonstrating organisation and planning in your approach 

to quality improvement. 

2 You need to know why you have chosen a particular tool 
and be able to justify it. 

3 A tool that is useful in one context may be useless in another. 

4 Different tools require different degrees of learner involvement. 

5 Provide evidence of learner involvement and its results. 

6 Evaluate the efficiency and effectiveness of the tool used. 
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Involving staff, learners 
and the community 
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One of an organisation's most important assets is an intangible one : 
its reputation. What makes a reputation? Who 'owns' it? How can 
it be improved? It could be argued that a provider's reputation 
resides with its 'customers' : staff, learners and the community. 

All of these are involved in the process of listening to learners. 

This chapter explains how. 



Staff 

It is important that staff at all levels 'own' the quality improvement 
system and are clear about their role in it. This can only be achieved 
through clear communication and staff development. One approach 
is to have a staff charter, making it clear what staff can expect from 
the process. 

Some quality manuals set out what individual staff members 
(and learners) should be doing at each stage of the process. 

The best manuals explain why it is important to listen to learners, 
and give clear guidance on when and how to do so. The manuals 
are backed up by staff development opportunities. 

It is important to build on existing resources. Many tutors and 
administrators already have an array of 'primary evidence' on 
learners' (and others') interactions with the service, for example 
statistical analyses of numbers of telephone calls taken or a 
daybook recording views. Encourage staff to ask one question 
a day - eg 'how did you hear about the service?' or 'what did 
you think of the brochure?' - and record responses. 
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Staff development could focus on looking at all the processes from 
a learner's perspective. 

Staff in other departments or organisations may also be able to help. 
They may have better communication than you with the target group, 
perhaps in terms of language, an established culture of trust or 
shared values. Team up with community development workers to get 
feedback on your service from people in areas of low participation, 
for example. Working with health professionals can lead to 
new provision and funding opportunities. Capitalise on the mutual 
interest across departments and organisations in engaging people 
in learning. 

The tutor in the classroom is well placed to listen to existing learners 
as a part of the learning process. Feedback should be planned 
for and recorded in a way that is appropriate to the course and 
can involve visual as well as written formats. Information technology 
courses might use an electronic voting system at the end of each 
session to influence the content of the next session, for example. 
Learners in residential or intensive education may participate in 
an introductory ’learning styles' workshop, so that the appropriate 
teaching techniques are subsequently used. This means of listening, 
allowing more ’reflective' learners to have more free time to absorb 
learning at their own pace, can affect retention. 



Learners 

To communicate with learners, and encourage their involvement, 
staff must understand the language and the culture of the learner. 
Staff need general communication skills and sometimes 
specific ones, such as : 

o languages other than English 

o Makaton or other systems designed for adults with 
learning difficulties 

® British Sign Language. 
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Communication may be influenced by the age, gender or perceived 
social class of the member of staff. For this reason some providers 
have trained other learners to obtain feedback and views from 
their peers. Such schemes have sometimes been run as widening 
participation courses in themselves and have an impact on 
community activism and personal development as well as 
needs analysis or evaluation. In fact, the best approaches 
do not separate the processes at all. 

Providers may establish an 'entitlement' (see the section on 
'Setting standards' under 'Learning effectively', above) as part of a 
quality policy published for learners. Versions of this may be adapted 
to meet the needs of specific groups. Once people are aware of 
their right to be involved, they are more likely to use it. Our right 
to learn (Jacobson 2000, 167-89), a pack aimed at adults 
with learning difficulties, defines an 'entitlement' in a way that 
is readily accessible by learners. 



The community 

You can use similar methods for involving the community. 

The best method is to work with existing community groups, 
such as regeneration partnerships, tenants' associations, 
drop-in centres and faith communities, to link a learning theme 
with the overall context of their work. 

It is important to build on existing expertise in the community. 
Some providers have commissioned community groups to canvass 
the views of those they come into contact with. This can create 
a 'win-win-win' situation: 

□ the provider obtains evidence and can create strategies 
to recruit more learners 

0 the potential learner is more likely to find a course to suit 
his or her needs 

b the community group receives additional funding to engage 
in an activity that complements its core work. 

Some ACL providers have worked alongside partners in the area 
(eg within local learning partnerships) to survey learners' and 
non-learners' views across a geographical area. 
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Seeing results 

Whatever the outcome of seeking feedback from learners or the 
community in general, all parties - including staff - need to see 
their efforts lead to something. 

This is not the same as saying : feedback must lead to change. 

That is not true. Feedback may tell the provider not to change 
approaches or curricula. The provider may not be able to address 
an issue because it is too expensive or it is a low priority, for example 
complaints about lack of car-parking space. 

Once you have fed back to the learner, there are three more things 
that you can do : 

d ask for views on the proposed actions 
0 ask what else could have been done 
eh thank people for their involvement. 

If listening to learners is about ownership, then the views of learner 
and the community need to be returned to them. 



Summary 

The process of listening to learners needs to involve, and be 
owned by, its contributors : staff, learners and the wider community. 



Key points 

1 Your reputation is 'owned' by your customers : staff, learners 
and the community. 

2 Involve staff by communicating clearly the reasons for 
listening to learners : why, what for, when and how. 

3 Share staff and resources with other organisations to 
listen to learners. 

4 Involve community groups to create a 'win-win-win' situation. 

5 Feed back to learners what you have done and what you intend to do. 
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Appendix i Listening frameworks 



Quality framework Listening to learners 



Charter Mark 'This is about consulting and involving users, 

staff and the wider community about how 
services can be improved and then effecting 
improvements' (DETR 2000, 17). 

More specifically, the Charter Mark provides 
a framework for a variety of techniques and 
approaches. This approach is about planning 
to get feedback, and assessing the cost and 
the impact of listening to service users. 



EFQM The model's 'results' criteria focus on 

Excellence Model what the organisation achieves. It looks at 

'people results' (staff), 'performance results', 
'society results' and 'customer results'. 
'Customer results' can also be used to meet 
the requirement in Best Value to measure 
user and citizen satisfaction. This does not 
mean it meets all the requirements of 
'consultation'. (See DETR 2000, 14.) 



Investors in People This model emphasises the role of staff within 
the organisation. The key role of front-line staff 
affects the quality of feedback received from 
learners and whether the organisation 
is perceived as open to suggestions. 

However, getting feedback from customers 
is not a key element of the scheme. 
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Quality framework Listening to learners 



ISO 9000 This model and preparation for assessment 

allows managers to think clearly and consis- 
tently about key business elements across 
their organisations. One of these is the 
key question : 'What do your customers want?' 
Applying this model systematically means 
that customer requirements are divided 
into two categories: 'defined' and 'implied'. 

The first is broken down into a detailed analysis 
of the product or service ; the second is about 
the 'overall perception of [the service] in the 
eyes of [the] customers'. This leads into the 
key area of customer service as part of the 
product being offered. Many in the education 
and training sector may feel uncomfortable 
with the terminology of 'requirements analysis', 
but the model does apply a useful way of looking 
clearly and logically at the process of interacting 
with learners. (See Munro-Faure et al. 1993, 
17-18.) 



PQASSO This framework gives explicit examples of 

evidence and stresses the importance of 
involvement at all stages of the operation, 
including the process of user involvement 
itself. The importance of acting on feedback 
and then communicating this to learners 
is highlighted. Record-keeping is given 
due emphasis, as is the link into strategic and 
operational plans. User and staff involvement 
is mentioned in a number of sections of the 
framework and is not isolated as just one part 
of the quality system. 
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